Appearance of Random Matrix Theory in deep learning

We investigate the local spectral statistics of loss surface Hessians artificial neural networks, where we discover excellent agreement with Gaussian Orthogonal Ensemble across several network architectures and datasets. These results shed new light on applicability Random Matrix Theory to modelling networks suggest a previously unrecognised role for it in study surfaces deep learning. Inspired by these observations, propose novel model true consistent our which allows Hessian densities rank degeneracy outliers, extensively observed practice, predicts growing independence gradients as function distance weight-space. further importance find, contrast previous work, that exponential hardness locating global minimum has practical consequences achieving state art performance.